Windows functional test #268

Sam071100 · 2021-09-07T09:37:31Z

Addresses Issue #27

darosior · 2021-09-07T09:46:38Z

Discussed this with @Sam071100 outside of Github, and we need to find a workaround for the OpenSSL-on-Windows issue: petertodd/python-bitcoinlib#262 .

The current explored option is to drop the python-bitcoinlib dependency, which we virtually only use for the RPC connection to bitcoind. This could be trivially replaced by HTTP requests to the endpoint.

I'm surprised i didn't have an issue to mark as closed now. Maybe mentioning revault#268 which is made largely easier with this.

darosior · 2021-12-18T09:34:55Z

@Sam071100 are you still planning to work on this?

darosior · 2021-12-18T10:05:31Z

Rebased and updated this PR.

danielabrozzoni · 2021-12-18T11:21:36Z

Some issues we have:

Sometimes the JSON RPC path is too long:

b"[1639825344][revaultd][INFO] Using Noise static public key: '3698d872edc4446733657398e3b4e9d11dab14cec0ac2f83558450246458fc4d'"
b"[1639825344][revaultd][DEBUG] Coordinator static public key: 'd91563973102454a7830137e92d0548bc83b4ea2799f1df04622ca1307381402'"
b'[1639825344][revaultd::daemon][INFO] Setting up database'
b'[1639825344][revaultd::daemon::database::actions][INFO] No database at "\\\\\\\\?\\\\C:\\\\Users\\\\Daniela\\\\AppData\\\\Local\\\\Temp\\\\revaultd-_6mjdeyh\\\\test_reorged_deposit_1\\\\revaultd\\\\regtest\\\\revaultd.sqlite3", creating a new one.'
b'[1639825344][revaultd::daemon][INFO] Setting up bitcoind connection'
b'[1639825344][revaultd::daemon][INFO] Starting JSONRPC server'
b'[1639825344][revaultd::daemon][ERROR] panic occurred at line 47 of file src\\daemon\\mod.rs: Some("Setting up JSONRPC server: Custom { kind: InvalidInput, error: \\"path must be shorter than SUN_LEN\\" }")'

setting the environment variable TEST_DIR helps.

We have some troubles with the JSON RPC (maybe windows: parse error of JSONRPC request #339 is related?). Calling getinfo works, as it doesn't have arguments, but calling listvaults with arguments throws:

(venv) C:\Users\Daniela\Downloads\revaultd-Windows_Functional_Test\revaultd-Windows_Functional_Test\tests>pytest -k test_listvaults
=================================================================================================================================================== test session starts ====================================================================================================================================================
platform win32 -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-0.13.1
rootdir: C:\Users\Daniela\Downloads\revaultd-Windows_Functional_Test\revaultd-Windows_Functional_Test, configfile: pyproject.toml
plugins: forked-1.4.0, timeout-1.3.4, xdist-1.31.0
collected 41 items / 40 deselected / 1 selected

test_rpc.py::test_listvaults
------------------------------------------------------------------------------------------------------------------------------------------------------ live log setup ------------------------------------------------------------------------------------------------------------------------------------------------------
INFO     root:bitcoind.py:82 BitcoinD started
FAILED                                                                                                                                                                                                                                                                                                                [100%]

========================================================================================================================================================= FAILURES =========================================================================================================================================================
_____________________________________________________________________________________________________________________________________________________ test_listvaults ______________________________________________________________________________________________________________________________________________________

revaultd_manager = <test_framework.revaultd.ManagerRevaultd object at 0x000001E6D6C57BB0>, bitcoind = <test_framework.bitcoind.BitcoinD object at 0x000001E6D6A1F280>

    def test_listvaults(revaultd_manager, bitcoind):
        res = revaultd_manager.rpc.call("listvaults")
        assert res["vaults"] == []

        # Send to a deposit address, we detect one unconfirmed vault
        amount_sent = 0.75
        addr = revaultd_manager.rpc.call("getdepositaddress")["address"]
        txid = bitcoind.rpc.sendtoaddress(addr, amount_sent)
        revaultd_manager.wait_for_log("Got a new unconfirmed deposit")
        vault_list = revaultd_manager.rpc.call("listvaults")["vaults"]
        assert len(vault_list) == 1
        assert vault_list[0]["status"] == "unconfirmed"
        assert vault_list[0]["txid"] == txid
        assert vault_list[0]["amount"] == amount_sent * 10 ** 8
        assert vault_list[0]["address"] == addr
        assert vault_list[0]["derivation_index"] == 0
        assert vault_list[0]["blockheight"] == 0
        assert vault_list[0]["funded_at"] is None
        assert vault_list[0]["secured_at"] is None
        assert vault_list[0]["delegated_at"] is None
        assert vault_list[0]["moved_at"] is None
        assert revaultd_manager.rpc.call("getinfo")["vaults"] == 1

        # Generate 5 blocks, it is still unconfirmed
        bitcoind.generate_block(5)
        assert (
            revaultd_manager.rpc.call("listvaults")["vaults"][0]["status"] == "unconfirmed"
        )

        # 1 more block will get it confirmed
        bitcoind.generate_block(1)
        revaultd_manager.wait_for_log(f"Vault at .*{txid}.* is now confirmed")
        vault = revaultd_manager.rpc.call("listvaults")["vaults"][0]
        assert vault["status"] == "funded"
        assert vault["funded_at"] is not None
        assert vault["secured_at"] is None
        assert vault["delegated_at"] is None
        assert vault["moved_at"] is None
        assert vault["blockheight"] == bitcoind.rpc.getblockcount() - 5

        # Of course, it persists across restarts.
        revaultd_manager.rpc.call("stop")
        revaultd_manager.proc.wait(TIMEOUT)
        revaultd_manager.start()
        vault_list = revaultd_manager.rpc.call("listvaults")["vaults"]
        assert len(vault_list) == 1
        assert vault_list[0]["status"] == "funded"
        assert vault_list[0]["txid"] == txid
        assert vault_list[0]["amount"] == amount_sent * 10 ** 8
        assert vault_list[0]["address"] == addr
        assert vault["funded_at"] is not None
        assert vault["secured_at"] is None
        assert vault["delegated_at"] is None
        assert vault["moved_at"] is None
        assert vault_list[0]["derivation_index"] == 0

        # And we can filter the result by status
>       vault_list = revaultd_manager.rpc.call("listvaults", [["unconfirmed"]])["vaults"]

test_rpc.py:98:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_framework\utils.py:385: in call
    resp = subprocess.check_output(
..\..\..\..\AppData\Local\Programs\Python\Python310\lib\subprocess.py:420: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
..\..\..\..\AppData\Local\Programs\Python\Python310\lib\subprocess.py:501: in run
    with Popen(*popenargs, **kwargs) as process:
..\..\..\..\AppData\Local\Programs\Python\Python310\lib\subprocess.py:966: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
..\..\..\..\AppData\Local\Programs\Python\Python310\lib\subprocess.py:1375: in _execute_child
    args = list2cmdline(args)
..\..\..\..\AppData\Local\Programs\Python\Python310\lib\subprocess.py:561: in list2cmdline
    for arg in map(os.fsdecode, seq):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

filename = ['unconfirmed']

    def fsdecode(filename):
        """Decode filename (an os.PathLike, bytes, or str) from the filesystem
        encoding with 'surrogateescape' error handler, return str unchanged. On
        Windows, use 'strict' error handler if the file system encoding is
        'mbcs' (which is the default encoding).
        """
>       filename = fspath(filename)  # Does type-checking of `filename`.
E       TypeError: expected str, bytes or os.PathLike object, not list

..\..\..\..\AppData\Local\Programs\Python\Python310\lib\os.py:822: TypeError
-------------------------------------------------------------------------------------------------------------------------------------------------- Captured stdout setup ---------------------------------------------------------------------------------------------------------------------------------------------------
Running tests in C:\Users\Daniela\RevaultdTests\revaultd-jo8mi550
-------------------------------------------------------------------------------------------------------------------------------------------------- Captured stderr setup ---------------------------------------------------------------------------------------------------------------------------------------------------
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
---------------------------------------------------------------------------------------------------------------------------------------------------- Captured log setup ----------------------------------------------------------------------------------------------------------------------------------------------------
INFO     root:bitcoind.py:82 BitcoinD started
------------------------------------------------------------------------------------------------------------------------------------------------- Captured stdout teardown -------------------------------------------------------------------------------------------------------------------------------------------------
Test failed, leaving directory 'C:\Users\Daniela\RevaultdTests\revaultd-jo8mi550\test_listvaults_1' intact
Leaving base dir 'C:\Users\Daniela\RevaultdTests\revaultd-jo8mi550' as it still contains ['test_listvaults_1']
===================================================================================================================================================== warnings summary =====================================================================================================================================================
tests/test_rpc.py: 4340 warnings
  C:\Users\Daniela\Downloads\revaultd-Windows_Functional_Test\revaultd-Windows_Functional_Test\tests\test_framework\utils.py:516: DeprecationWarning: notifyAll() is deprecated, use notify_all() instead
    self.logs_cond.notifyAll()

-- Docs: https://docs.pytest.org/en/stable/warnings.html
================================================================================================================================================= short test summary info ==================================================================================================================================================
FAILED test_rpc.py::test_listvaults - TypeError: expected str, bytes or os.PathLike object, not list
===================================================================================================================================== 1 failed, 40 deselected, 4340 warnings in 27.04s =====================================================================================================================================

darosior added a commit to darosior/revaultd that referenced this pull request Dec 4, 2021

qa: finally get rid of python-bitcoinlib

7231b5d

I'm surprised i didn't have an issue to mark as closed now. Maybe mentioning revault#268 which is made largely easier with this.

darosior added a commit to darosior/revaultd that referenced this pull request Dec 6, 2021

qa: finally get rid of python-bitcoinlib

66a2e8e

I'm surprised i didn't have an issue to mark as closed now. Maybe mentioning revault#268 which is made largely easier with this.

Sam071100 added 2 commits December 18, 2021 10:40

Adds portability in order to run the functional tests under Windows

8086d31

Updated Readme

269b0af

darosior force-pushed the Windows_Functional_Test branch from 69bf411 to 269b0af Compare December 18, 2021 10:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Windows functional test #268

Windows functional test #268

Sam071100 commented Sep 7, 2021

darosior commented Sep 7, 2021

darosior commented Dec 18, 2021

darosior commented Dec 18, 2021

danielabrozzoni commented Dec 18, 2021

Windows functional test #268

Are you sure you want to change the base?

Windows functional test #268

Conversation

Sam071100 commented Sep 7, 2021

darosior commented Sep 7, 2021

darosior commented Dec 18, 2021

darosior commented Dec 18, 2021

danielabrozzoni commented Dec 18, 2021